Caching content securely within an edge environment

ABSTRACT

A technique to cache content securely within edge network environments, even within portions of that network that might be considered less secure than what a customer desires, while still providing the acceleration and off-loading benefits of the edge network. The approach ensures that customer confidential data (whether content, keys, etc.) are not exposed either in transit or at rest. In this approach, only encrypted copies of the customer&#39;s content objects are maintained within the portion of the edge network, but without any need to manage the encryption keys. To take full advantage of the secure content caching technique, preferably the encrypted content (or portions thereof) are pre-positioned within the edge network portion to improve performance of secure content delivery from the environment.

BACKGROUND Technical Field

This application relates generally to secure network-basedcommunications using cryptographic protocols such as Transport LayerSecurity (TLS).

Brief Description of the Related Art

Transport Layer Security (TLS) and its predecessor, Secure Sockets Layer(SSL), are cryptographic protocols that provide Internet communicationsecurity. They use asymmetric cryptography for authentication and keyexchange, symmetric encryption for confidentiality, and messageauthentication codes for message integrity. TLS/SSL is initialized at asession layer then works at a presentation layer. In particular, firstthe session layer has a handshake using an asymmetric cipher toestablish cipher settings and a shared key for that session. Thereafter,a presentation layer encrypts the rest of the communication using asymmetric cipher and that session key. In both models, TLS and SSL workon behalf of the underlying transport layer, whose segments carryencrypted data. TLS is an IETF standards track protocol, defined in RFC5246 and RFC 6176.

Distributed computer systems are well-known in the prior art. One suchdistributed computer system is a “content delivery network” (CDN) or“overlay network” that is operated and managed by a service provider.The service provider typically provides the content delivery service onbehalf of third parties (customers) who use the service provider'sshared infrastructure. A distributed system of this type typicallyrefers to a collection of autonomous computers linked by a network ornetworks, together with the software, systems, protocols and techniquesdesigned to facilitate various services, such as content delivery, webapplication acceleration, or other support of outsourced origin siteinfrastructure. A CDN service provider typically provides servicedelivery through digital properties (such as a website), which areprovisioned in a customer portal and then deployed to the network. Adigital property typically is bound to one or more edge configurationsthat allow the service provider to account for traffic and bill itscustomer.

For a traditional RSA-based TLS session, the two sides of a connectionagree upon a “pre-master secret” (PMS) which is used to generate theparameters for the remainder of the session. Typically, the two sidesuse RSA asymmetric encryption to establish the pre-master secret withoutexchanging the actual value in plaintext. In operation, the SSL clientgenerates the pre-master secret and encrypts it with the TLS server'spublicly available RSA key. This generates an encrypted pre-mastersecret (ePMS), which is then provided to the TLS server. The TLS serverhas a private decryption key, which is then used to decrypt theencrypted pre-master secret. At this point, both the client and theserver have the original pre-master secret and can use it to generatethe symmetric key used for actual encrypted and secure data exchange.Decrypting the encrypted pre-master secret is the only time in the TLSconnection that the private key is needed. This decryption occurs at aso-called TLS termination point. Where a CDN is used to facilitatedelivery of secure content, typically the TLS termination point will belocated in the CDN.

Some CDN customers are not comfortable sharing their private TLS (RSA,DSA, etc.) keys with the CDN service provider. Further, some customersmay require an additional caveat that certain data and requests never bedecrypted by the CDN at any point in any transaction, and that the datatransmitted by the CDN on behalf of the customer is provably andverifiably authentic.

BRIEF SUMMARY

This disclosure describes a technique to cache content securely withinedge network environments, even within portions of that network thatmight be considered less secure than what a customer desires, whilestill providing the acceleration and off-loading benefits of the edgenetwork. The approach ensures that customer confidential data (whethercontent, keys, etc.) are not exposed either in transit or at rest. Inthis approach, only encrypted copies of the customer's content objectsare maintained within the portion of the edge network, but without anyneed to manage the encryption keys. According to another aspect, and totake full advantage of the secure content caching technique, preferablythe encrypted content (or portions thereof) are pre-positioned withinthe edge network portion to improve performance of secure contentdelivery from the environment.

The foregoing has outlined some of the more pertinent features of thesubject matter. These features should be construed to be merelyillustrative. Many other beneficial results can be attained by applyingthe disclosed subject matter in a different manner or by modifying thesubject matter as will be described.

BRIEF DESCRIPTION OF THE DRAWINGS

For a more complete understanding of the subject matter and theadvantages thereof, reference is now made to the following descriptionstaken in conjunction with the accompanying drawings, in which:

FIG. 1 is a block diagram illustrating a known distributed computersystem configured as a content delivery network (CDN);

FIG. 2 is a representative CDN edge machine configuration; and

FIG. 3 is a representative active TLS session among a client, an edgeserver, and an origin server;

FIG. 4 depicts a multi-layered edge network in which the subject matterof this disclosure may be implemented;

FIG. 5 depicts how an EdgeLocker acts as an intermediary between arequesting end user client machine and an upstream edge server globalhost (ghost) process according to this disclosure;

FIG. 6 depicts the EdgeLocker-ghost server interaction in more detail;and

FIG. 7 depicts a TLS record for an encrypt-then-MAC operation accordingto this disclosure.

DESCRIPTION

In a known system, such as shown in FIG. 1 , a distributed computersystem 100 is configured as a content delivery network (CDN) and isassumed to have a set of machines 102 a-n distributed around theInternet. Typically, most of the machines are servers located near theedge of the Internet, i.e., at or adjacent end user access networks. Anetwork operations command center (NOCC) 104 manages operations of thevarious machines in the system. Third party sites, such as web site 106,offload delivery of content (e.g., HTML, embedded page objects,streaming media, software downloads, and the like) to the distributedcomputer system 100 and, in particular, to “edge” servers. Typically,content providers offload their content delivery by aliasing (e.g., by aDNS CNAME) given content provider domains or sub-domains to domains thatare managed by the service provider's authoritative domain name service.End users that desire the content are directed to the distributedcomputer system to obtain that content more reliably and efficiently.Although not shown in detail, the distributed computer system may alsoinclude other infrastructure, such as a distributed data collectionsystem 108 that collects usage and other data from the edge servers,aggregates that data across a region or set of regions, and passes thatdata to other back-end systems 110, 112, 114 and 116 to facilitatemonitoring, logging, alerts, billing, management and other operationaland administrative functions. Distributed network agents 118 monitor thenetwork as well as the server loads and provide network, traffic andload data to a DNS query handling mechanism 115, which is authoritativefor content domains being managed by the CDN. A distributed datatransport mechanism 120 may be used to distribute control information(e.g., metadata to manage content, to facilitate load balancing, and thelike) to the edge servers.

As illustrated in FIG. 2 , a given machine 200 comprises commodityhardware (e.g., an Intel Pentium processor) 202 running an operatingsystem kernel (such as Linux or variant) 204 that supports one or moreapplications 206 a-n. To facilitate content delivery services, forexample, given machines typically run a set of applications, such as anHTTP proxy 207 (sometimes referred to as a “global host” process), aname server 208, a local monitoring process 210, a distributed datacollection process 212, and the like. For streaming media, the machinetypically includes one or more media servers, such as a Windows MediaServer (WMS) or Flash server, as required by the supported mediaformats.

A CDN edge server is configured to provide one or more extended contentdelivery features, preferably on a domain-specific, customer-specificbasis, preferably using configuration files that are distributed to theedge servers using a configuration system. A given configuration filepreferably is XML-based and includes a set of content handling rules anddirectives that facilitate one or more advanced content handlingfeatures. The configuration file may be delivered to the CDN edge servervia the data transport mechanism. U.S. Pat. No. 7,111,057 illustrates auseful infrastructure for delivering and managing edge server contentcontrol information, and this and other edge server control informationcan be provisioned by the CDN service provider itself, or (via anextranet or the like) the content provider customer who operates theorigin server.

The CDN may include a storage subsystem, such as described in U.S. Pat.No. 7,472,178, the disclosure of which is incorporated herein byreference.

The CDN may operate a server cache hierarchy to provide intermediatecaching of customer content; one such cache hierarchy subsystem isdescribed in U.S. Pat. No. 7,376,716, the disclosure of which isincorporated herein by reference.

The CDN may provide secure content delivery among a client browser, edgeserver and customer origin server in the manner described in U.S.Publication No. 20040093419. Secure content delivery as describedtherein enforces SSL-based links between the client and the edge serverprocess, on the one hand, and between the edge server process and anorigin server process, on the other hand. This enables an SSL-protectedweb page and/or components thereof to be delivered via the edge server.To enhance security, the service provider may provide additionalsecurity associated with the edge servers. This may include operatingsecure edge regions comprising edge servers located in locked cages thatare monitored by security cameras.

As an overlay, the CDN resources may be used to facilitate wide areanetwork (WAN) acceleration services between enterprise data centers(which may be privately-managed) and third party software-as-a-service(SaaS) providers.

In a typical operation, a content provider identifies a content providerdomain or sub-domain that it desires to have served by the CDN. The CDNservice provider associates (e.g., via a canonical name, or CNAME) thecontent provider domain with an edge network (CDN) hostname, and the CDNprovider then provides that edge network hostname to the contentprovider. When a DNS query to the content provider domain or sub-domainis received at the content provider's domain name servers, those serversrespond by returning the edge network hostname. The edge networkhostname points to the CDN, and that edge network hostname is thenresolved through the CDN name service. To that end, the CDN name servicereturns one or more IP addresses. The requesting client browser thenmakes a content request (e.g., via HTTP or HTTPS) to an edge serverassociated with the IP address. The request includes a host header thatincludes the original content provider domain or sub-domain. Uponreceipt of the request with the host header, the edge server checks itsconfiguration file to determine whether the content domain or sub-domainrequested is actually being handled by the CDN. If so, the edge serverapplies its content handling rules and directives for that domain orsub-domain as specified in the configuration. These content handlingrules and directives may be located within an XML-based “metadata”configuration file.

More generally, the techniques described herein are provided using a setof one or more computing-related entities (systems, machines, processes,programs, libraries, functions, or the like) that together facilitate orprovide the described functionality described above. In a typicalimplementation, a representative machine on which the software executescomprises commodity hardware, an operating system, an applicationruntime environment, and a set of applications or processes andassociated data, that provide the functionality of a given system orsubsystem. As described, the functionality may be implemented in astandalone machine, or across a distributed set of machines. Thefunctionality may be provided as a service, e.g., as a SaaS solution.

Content Delivery Network with TLS Support

As used herein, an “edge server” refers to a CDN (overlay network) edgemachine. For a given customer, the CDN service provider may allow a TCPconnection to originate from a client (e.g., an end user browser, ormobile app) and connect to an edge machine representing the customer ona virtual IP address (VIP) assigned to the customer, or a general VIPthat allows for discovery of the intended customer. For purposes of thisdisclosure, it is assumed that this edge machine does not necessarilyhave the customer's private key or the customer's certificate.

As illustrated in FIG. 3 , in the typical interaction scenario, an enduser client browser or mobile app 300 is associated with a customerorigin server (or “origin”) 302 via the intermediary of an overlaynetwork edge machine server instance 304 (sometimes referred to as an“edge server”). The terms “origin” or “edge” are not intended to belimiting.

The following provides details regarding the TLS handshake between theclient 300 and the origin 302. The reader's familiarity with the TLSSpecification is presumed. The edge machine server instance 304 passeshandshake messages through directly to the origin 302, and vice versa.Once the handshake is complete, the origin 302 and client 300 will havenegotiated a Pre-Master Secret and exchange random number values for thesession. According to the TLS 1.2 Specification Section 8.1, each sidewill have converted its Pre-Master Secret into a Master Secret using anagreed-upon pseudo-random function (PRF), such as an HMAC variant. TheTLS 1.2 Specification Section 6.3 notes that this Master Secret is thenconverted into a larger key block, which is then used to calculate thefollowing TLS items: client_write_MAC_key (the key that be used as inputto the client's sent data message authentication codes (MACs)), whereineach TLSCipherText record has a MAC that verifies the data in the recordis authentic and unchanged); server_write_MAC_key (the key that will beused as input to the server's sent data MACs); client_write_key (the keythat will be used for the agreed-upon bulk encryption cipher for clientsent data; and server_write_key (the key that will be used for theagreed-upon bulk encryption cipher for server sent data). Other itemsmay be calculated but, for purposes of this protocol, are not relevant.

The following provides details regarding the handling of encryptedrequests and responses in this known approach. The requests areencrypted using the parameters from the TLS handshake described above.In a typical operation, the CDN edge server 304 receives the TLS records(that represent the request) from the client 300 and simply forwardsthem along to the origin 302, unable to read them due to the encryption.When the response data (from the origin) is private and should not beshared with the CDN, the origin 300 responds back to the edge server 304using the conventional TLS mechanisms. The edge server 304 receives thisdata and simply passes it through to the client 300, again unable toread it due to the encryption. In other words, the edge server 304 issimply acting as a pass-through TCP proxy. It can neither decrypt datanor contribute data to the TLS session.

FIG. 4 depicts a multi-layered operating environment in which thesubject matter of this disclosure may be implemented. A first layercomprises two (2) types of servers, namely, secure cache servers 402,and shareable cache servers 404. As will be described in more detailbelow, the “secure cache servers” 402 are used to facilitate certaintechniques of this disclosure. A secure cache server 402 is sometimesreferred to herein as “EdgeLocker,” although this nomenclature is notintended to be limiting.

A second layer comprises two (2) additional types of servers, namely,delegated content servers 406, and ghost servers 408.

Although the delegated content server 406 is shown as distinct from theghost server 408, this is not a requirement, as a ghost server mayoperate in the role of the delegated content server (as will bedescribed below by way of example).

A third layer comprises on-premises gateways or gateway servers 410,which are typically located behind a corporate (CDN customer) firewall412. As further depicted, it is presumed that end user client machines401 desire to interact with the content provider origin 403. As alsodepicted, a crypto-server machine (or set of machines) 414 may be usedto maintain private keys for TLS session establishment. Thecrypto-server and its operation within the context of a typicalTLS-based implementation is described in U.S. Publication No.US2015/0067338.

The various servers may operate within the context of different securitymodels, each providing a different level of key exposure. Thus, forexample, the ghost servers 408 may operate in secure physicalenvironments (e.g., with appliances located in secure monitored cages),and host customer private keys. The delegated content servers 406 mayoperate with or outside of such secure physical environment but arepresumed to at least host TLS session keys (these servers would theninteract with the crypto server 414 for TLS termination). The shareablecache servers typically store un-encrypted data that are later validatedand encrypted when serving to the end user client machines 401. Thisun-encrypted data can be shared among different TLS sessions, thusimproving cache efficiency. A shareable cache server 404 may interactwith a delegated content server 406 (or some other CDN provider-managedserver) using the TLS splicing approach described in U.S. Pat. No.9,137,218; in such circumstances, the key exposure is limited to theserver write encryption key.

According to this disclosure, and as will be described below, preferablythe secure cache server (the EdgeLocker) has no keys; thus, it has nokey exposure per se.

Preferably, the servers 402 and 404 in the first layer provide datacaching, operating on TLS records, and preferably they not perform anyHTTP request processing. Instead, servers—such as the EdgeLockers402—preferably rely on servers in other layers for the HTTP requestsprocessing, and they take instructions from those other servers. Thesecond layer of servers, in contrast, preferably handle the HTTP requestprocessing, including the standard HTTP response caching. They can beused to serve end user clients directly and, as will be seen, they alsocan be used to assist the EdgeLockers to serve the requests (inaccordance with this disclosure). Ghost servers typically are entrustedto be a full proxy of the content provider origin.

As noted above, typically the on-premises gateways are not part of thephysical “edge” but, rather, are located behind a customer firewall.These servers are desirable in certain use cases. For customers that donot want their private keys anywhere outside of their firewalls,however, the gateways may also provide the crypto-server function. Forcustomers that do not want any content cached un-encrypted anywhere(except behind the firewall), the gateways may also act as contentservers for the secure cache servers 402.

Caching and Pre-Populating Content Securely Within a Multi-Layered EdgeEnvironment

FIG. 5 depicts a preferred implementation of the techniques of thisdisclosure.

Within the context of this disclosure, and relative to the ghost server(or the second layer generally), the EdgeLocker is considered to belocated within a portion of the edge network that is considered to be“further” away. There may be one or more reasons (or use cases) forthis, e.g., the EdgeLocker may be deeply deployed (or, at least, moredeeply deployed relative to a ghost server) within an end user networksuch that it is closer to the requesting end users than, say, are theghost servers. In this context, the EdgeLocker is sometimes referred toas located within an “outer edge” wherein the ghost server is locatedwithin an “inner edge.” The terms “outer” and “inner” as used herein arenot necessarily meant to imply some actual physical notion (althoughthey may), but rather a difference in the sensitivity of those networkelements (e.g., to key exposure, or even content exposure). As such, theEdgeLocker is considered to be more sensitive to key and contentexposure that, say, the ghost servers or other servers in the CDN.

According to this disclosure, EdgeLocker preferably keeps only encryptedcopies of the objects in the network, but without the need to manage theencryption keys. The solution leverages the bulk encryption already doneby the TLS layer and caches the encrypted TLS segments in the network.Because end-user clients already have the bulk encryption keys, however,no additional key management is needed in this approach.

As depicted in FIG. 5 , a multi-layered edge network 500 comprises amore secure inner edge 504, and a relatively less secure outer edge 502.The inner edge runs ghost servers 505, preferably operating a full-stackthat handles the TLS termination and the HTTP requests; in contrast, theouter edge preferably operates at the transport layer providing DDoSprotection, data caching and serving the cached data. The EdgeLockers503 run in the outer edge. The outer edge 502 caches the TLS encryptedfragments, if needed, as instructed by the inner edge 504. Preferably,the inner edge also coordinates with the outer edge to ensure thecommunication with the end-user clients conforms to the protocolstandards.

As noted above, there are several use cases for the solution. Theseinclude an extension to a CDN secure delivery service to improvecapacity (and likely performance) for “secure-grade” delivery withoutracking up the cost of managing additional secure edge regions. In thiscontext, the extension acts as the outer edge to provide increasedgeo-footprint, DDoS protection capacity and offloading. Another use caseis the on-premises behind-the-firewall gateways for customers desiringto control all of their web resources without having any un-encrypteddata on the content delivery network. In this context, the gateways actas the inner edge to the Edge Lockers. Another use case is for carriernetwork deployments, which can also provide the outer edge for securecontent delivery.

As depicted in FIG. 5 , the multi-layer edge network 500 is used forserving secure traffic. The end-user clients 501 make requests in aconventional manner. The EdgeLockers 503 in the outer edge 502 work withthe ghost servers 505 in the inner edge 504 to serve the requests.Preferably, and as depicted, the ghost servers 505 maintain persistentcontrol channels 506 with the EdgeLockers 503 to coordinate the datacaching and serving of the cached data.

The following is a representative workflow description. It is assumedthat other CDN functions (e.g., DNS region/server selection) areimplemented to identify the particular EdgeLocker instance and ghostserver that are involved in the communication. These are well-known CDNfunctions, such as described in U.S. Pat. No. 6,108,703.

Each of the EdgeLocker and the ghost server are implemented as one ormore software processes or programs executed in one or more hardwareprocessors. An EdgeLocker may be implemented in a standalone machine,e.g., a rack-based appliance. An EdgeLocker “instance” executes in amachine that typically is distinct from a “ghost server” instance thatexecutes in a different machine.

TLS Session Establishment and Request Processing

1. End-user client makes a request for a TLS session, either a new oneor a resumed one, to the EdgeLocker.

2. The EdgeLocker checks the request packets against a set of rules toensure the request is unlikely to be an attack. Such rules could includepre-configured firewall rules, client reputation rules, TLS sessionnegotiation rate limiting rules, and etc.

3. If the request is valid, it is forwarded to the ghost server.

4. The ghost server goes through the TLS session negotiation processwith the end-user client, with the EdgeLocker forwarding the packets.Because the EdgeLocker has no knowledge of any private keys, it does notobtain any session keys. However, the EdgeLocker could examine thepackets to ensure the TLS state changes are valid to protect againstattacks on the TLS negotiation process.

5. During the TLS session negotiation, the ghost server checks if theclient supports encrypt-then-mac (per the RFC7366 standard), and savesit for the later HTTP request processing. If the client does not havethe RFC7366-compliant support, the requests are processed in aconventional manner (as the ghost server normally would). Otherwise, therequests are processed as described in the paragraphs below.

6. After the TLS session is established, the end-user client sends theHTTP request, over the TLS session.

7. The EdgeLocker again forwards the packets to the ghost server.

8. The ghost server processes the request, preferably using a standardghost server HTTP processing pipeline. The request either could beserved from the ghost server cache, or by forwarding to the cacheparent, the origin, or the like, all as well-known.

Non-Cacheable Response Handling

1. If the response is non-cacheable, the ghost server just sends theresponse.

2. The EdgeLocker forwards the response to the end-user client.

Cacheable Response Handling

1. If the response is cacheable, the ghost server communicates the factto the EdgeLocker and expects a response from the EdgeLocker with theinformation on data it cached for the object.

2. If the EdgeLocker does not have the data cached, the ghost serversends the response.

3. The EdgeLocker stores the response and then forwards it on to therequesting end user client.

4. If the EdgeLocker has the data, the ghost server still generates theTLS records, but without sending them. Instead, the ghost server sendsthe updated information on the TLS records through the control channelto the EdgeLocker. The updated information has a much smaller size thanthe TLS records themselves.

5. The EdgeLocker uses that information, along with its cached data forthe object, to construct the TLS records, and it sends the TLS recordsto the end-user client.

Control Channel Interface Description

The ghost server and EdgeLocker preferably communicate over the controlchannel to coordinate the processing. The interactions preferably areinitiated from the ghost server, even though the connections areestablished from the EdgeLocker, preferably during its start up. In thismanner, the EdgeLocker is much more lightweight than the ghost server.

A preferred wire protocol uses the Google protobuf. One format of themessages is described below.

Request Number:

Each request message includes a request number that is also included inthe response, to correlate the request and response.

Cache Data Query Message:

The message from the ghost server querying the cached data preferablyincludes the following fields:

-   -   sessionid: the id uniquely identifies the set of session keys.        Note that the sessionid also captures the “version” of the bulk        encryption keys, in addition to the TLS session identifier.    -   docid: an id uniquely identifies the HTTP object being served.        Note that the docid also captures the “version” of the object.

In response, the message from the EdgeLocker preferably includes thefollowing fields:

-   -   ack: 1 if it has the data in cache, 2 otherwise, or other error        codes.        Cache Data Update Message:

The message from the ghost server updating the cached data preferablyincludes the following fields:

-   -   sessionid: the id uniquely identifies the set of session keys.    -   docid: an id uniquely identifies the HTTP object being served.    -   recs: list of data on the TLS records, where each entry includes        the following        -   seqnum: the new sequence number of the TLS record.        -   hash: the HMAC value.

In response, the message from the EdgeLocker preferably includes thefollowing fields:

-   -   ack: 1 for success or the error code otherwise.

Preferably, the control channel is a regular TLS connection. Inaddition, link between the ghost server and the EdgeLocker issemi-private, network-layer protection can also be enforced.

As a skilled person will appreciate, and because the data are encryptedwith the TLS session bulk encryption key, the cached data is only validfor the duration of the session. This is likely less efficient than ifthe plain text objects were to be cached. Therefore, if the use caseallows the caching of plain text objects, the above-described mechanismmay be by-passed (at least with respect to those objects). Other usecases, such as the REST API services (where the requests come from otherweb applications) may have long-standing sessions and thus use theapproach.

Of course, with multiple end-user clients using different TLS sessions,multiple encrypted copies need to be stored at the edge, even for thesame plain text object. Even so, the technique herein provides benefits,at least in part because the content to be served are cached at theedge. Keeping multiple copies of an object, in multiple-regions, is howCDNs provide the acceleration and content-origin offloading benefits.

Due to the nature of the tasks carried out by EdgeLockers (e.g., no HTTPprocessing or encryption), the hardware profile typically differs fromthat used to support other servers. As compared to ghost servers, forexample, these servers do not need much computing power, although theydo require more storage space to cache the data from the multiple TLSsessions.

FIG. 6 depicts the interaction between the EdgeLocker 600 and the ghostserver 602 in more detail. In general, the ghost server is considered tobe upstream of the EdgeLocker. In this approach, and as depicted theouter edge (the EdgeLocker 600) is caching the encrypted fragment 605,and the inner edge (the ghost server 602) is sending the updatedsequence number and the MAC (message authentication code) that resultsfrom having the updated sequence number and encrypted fragment. Asnoted, the EdgeLocker caches the encrypted TLS fragment. The ghostserver sends an instruction to the EdgeLocker to serve the encrypted TLSfragment and, in so doing, the ghost server provides the sequence numberfor the fragment and the MAC needed to enable the EdgeLocker toconstruct the TLS response appropriately. The EdgeLocker is able toconstruct and serve the TLS response 607 despite having only anencrypted fragment and no session-scope secret information about thesession, i.e., no keys.

FIG. 7 depicts a preferred structure for a TLS record constructed by theEdgeLocker. As can be seen, the EdgeLocker preferably implements an“encrypt-then-MAC” operation (as opposed to the MAC-then-encrypt in theRFC standard).

According to this disclosure, and as FIG. 7 illustrates, the EdgeLockerdoes not get any key information (e.g., the private D-H signing key, theserver write key, etc.) and stores objects encrypted. Usually this wouldmean that the server could not send a proper TLS record because it isotherwise unable to fold the MAC inside the encryption layer. To addressthis, the MAC is delivered outside of the encryption (see, e.g., FIG. 7), preferably leveraging the IETF extension in RFC 7366, but in adifferent manner). Thus, content fragment can remain encrypted on theouter edge, and the MAC can be sent from the upstream server (e.g.,ghost, the origin, etc.) at the time it is needed. The origin servertypically sends an identifier for the fragment and the MAC.

Pre-population may be used for any type of server, including theon-premises box. The inner edge can also execute pre-fetch operations,and push to the outer edge. A pre-fetching approach as described in U.S.Pat. No. 8,447,837 may be used for this purpose.

The mechanism described above enables the CDN to cache content that isnot currently cached at the edge, e.g., due to security reasons. Itapplies to content that is not changed during a session. The followingsection describes an extension that provides a mechanism to improve theperformance of serving the secure content when it does change during asession. The notion here is to “pre-position” the content to the edge.In one embodiment, the technique is applied to changed objects that arealready securely cached at the edge. In another embodiment, the approachis applied to objects that are yet to be requested, e.g., based onaccess patterns of the content. For example, requests to a given URL(such as a homepage) might be seen by the system to always involvefetching the homepage of the site right after the TLS sessionestablishment. In such circumstance, the content for the homepage ispre-positioned at the edge before the HTTP request comes in so that theoverall performance (using the EdgeLocker approach described) isimproved. As in the caching approach, preferably a multi-layer edgenetwork is used, with the inner edge handling the TLS termination andHTTP request handling, and the outer edge handling the data cache. Inthis case, content is pre-positioned in the cache to improve overallperformance.

The following is the preferred workflow for the pre-positioningapproach:

TLS Session Establishment and Request Processing

1. End-user client makes a request for a TLS session, either a new oneor a resumed one, to the EdgeLocker.

2. The EdgeLocker checks the request packets against a set of rules toensure the request is unlikely to be an attack. Such rules could includepre-configured firewall rules, client reputation rules, TLS sessionnegotiation rate limiting rules, and etc.

3. If the request is valid, it is forwarded to the ghost server.

4. The ghost server goes through the TLS session negotiation processwith the end-user client, with the EdgeLocker forwarding the packets.Because the EdgeLocker has no knowledge of any private keys, it does notobtain any session keys. However, the EdgeLocker could examine thepackets to ensure the TLS state changes are valid to protect againstattacks on the TLS negotiation process.

5. During the TLS session negotiation, the ghost server checks if theclient supports encrypt-then-mac (per the RFC7366 standard), and savesit for the later HTTP request processing. If the client does not havethe RFC7366-compliant support, the requests are processed in aconventional manner (as the ghost server normally would). Otherwise, therequests are processed as described in the paragraphs below.

Pre-Positioning Initial Content

1. For the objects that are designated by the content provider forinitial pre-positioning, the ghost server sends the encrypted objects tothe EdgeLocker, right after TLS session establishment, before the HTTPrequest comes in, through an auxiliary data channel (which differs fromthe control channel previously described).

2. The EdgeLocker stores the data from response, but without forwardingit to the end-user client.

3. The end-user client sends the HTTP request over the TLS session.

4. The EdgeLocker forwards the packets to the ghost server.

5. The ghost server processes the request and generates the TLS recordsof the response, but without sending them. Instead, it sends the updatedinformation on the TLS records, which has a much smaller size than theTLS records themselves, through the control channel to the EdgeLocker.

6. The EdgeLocker uses that information, along with its cached data forthe object, to construct the TLS records, and sends the TLS records tothe end-user client.

Pre-Positioning Changed Content

1. Similar to pre-positioning initial content, the ghost server can alsopush updated objects that are cached by the EdgeLocker before theend-user client requests them.

Auxiliary Data Channel Interface Description

The ghost server pushes the pre-positioned data to EdgeLocker over theauxiliary data channel. Preferably, the interactions are initiated fromthe ghost server, even though the connections are established from theEdgeLocker, preferably during its start up. The reasons for thisadditional channel, instead of the control channel, are as follows.

1. Because the data is already encrypted, an un-encrypted channel can beused, to reduce the overhead.

2. Using a separate channel avoids possible congestion that could leadto poor performance if such data is sent over the control channel.

Preferably, the wire protocol uses Google protobuf. The format of themessages is described below.

Request Number:

Each request message includes a request number that is also included inthe response, to correlate the request and response.

Cache Data Push Message:

The message from the ghost server pushing the data preferably includesthe following fields:

-   -   sessionid: the id uniquely identifies the set of session keys.        Note that the sessionid also captures the “version” of the bulk        encryption keys, in addition to the TLS session identifier.    -   docid: an id uniquely identifies the HTTP object being served.        Note that the docid also captures the “version” of the object.    -   recs: list of TLS records.

In response, the message from the EdgeLocker preferably includes thefollowing fields:

-   -   ack: 1 for success or the error code otherwise.

The EdgeLocker message protocol includes the session identifier toaddress the situation when there is a session resumption; in this case,the ghost server needs to be able to tell the Edge Locker which versionof the encrypted fragment to serve (i.e., to serve the version that isencrypted with the current session keys in the session, not the priorones).

Thus, in the pre-positioning (or pre-population) approach, a request fora first object is received at the upstream second server. The request isdelivered to the upstream second server by a first server. The secondserver is assumed in this example to have previously provided to thefirst server one or more second objects required by (or associated with)the first object. Objects are stored in the first server as encryptedTLS fragments. As required, the second server sends instruction(s) tothe first server to serve the encrypted TLS fragments for both the firstobject and the one or more second objects that have been pre-positionedin the first server in accordance with the disclosure.

TLS as implemented herein typically is TLS 1.2, and the particularcipher suite used in TLS is not a limitation. The approach may be usedwith advanced cipher suites, such as GCM AEAD ciphers. The references toTLS (or particular cipher suites therein) are merely exemplary. Thetechniques described herein may be generalized for use with anycryptographic protocol that uses both (i) asymmetric cryptography toexchange unidirectional symmetric keys, and (ii) separate authenticationcode keys. Thus, the techniques may be used for various SSL versions,next-generation TLS, and other such protocols.

The reference to HTTP-based requests and responses is merely exemplaryas well.

The techniques described above are not limited to the intermediary beingan edge server within an overlay network; thus, the above methods shouldbe generalized with respect to any third party entity (system, device,machine, program, process, execution thread, etc.) and not just the CDNedge server as described.

While the above describes a particular order of operations performed bycertain embodiments of the invention, it should be understood that suchorder is exemplary, as alternative embodiments may perform theoperations in a different order, combine certain operations, overlapcertain operations, or the like. References in the specification to agiven embodiment indicate that the embodiment described may include aparticular feature, structure, or characteristic, but every embodimentmay not necessarily include the particular feature, structure, orcharacteristic.

While the disclosed subject matter has been described in the context ofa method or process, the subject disclosure also relates to apparatusfor performing the operations herein. This apparatus may be speciallyconstructed for the required purposes, or it may comprise ageneral-purpose computer selectively activated or reconfigured by acomputer program stored in the computer. Such a computer program may bestored in a computer readable storage medium, such as, but is notlimited to, any type of disk including an optical disk, a CD-ROM, and amagnetic-optical disk, a read-only memory (ROM), a random access memory(RAM), a magnetic or optical card, or any type of media suitable forstoring electronic instructions, and each coupled to a computer systembus.

While given components of the system have been described separately, oneof ordinary skill will appreciate that some of the functions may becombined or shared in given instructions, program sequences, codeportions, and the like.

Preferably, the functionality is implemented in an application layersolution, although this is not a limitation, as portions of theidentified functions may be built into an operating system or the like.

The functionality may be implemented with other application layerprotocols besides HTTP and HTTPS, such as SSL VPN, or any other protocolhaving similar operating characteristics.

There is no limitation on the type of computing entity that mayimplement the client-side or server-side of the connection. Anycomputing entity (system, machine, device, program, process, utility, orthe like) may act as the client or the server. There is no limitation onthe type of computing entity that may implement the client-side orserver-side of the connection. Any computing entity (system, machine,device, program, process, utility, or the like) may act as the client orthe server.

While given components of the system have been described separately, oneof ordinary skill will appreciate that some of the functions may becombined or shared in given instructions, program sequences, codeportions, and the like. Any application or functionality describedherein may be implemented as native code, by providing hooks intoanother application, by facilitating use of the mechanism as a plug-in,by linking to the mechanism, and the like.

Because the EdgeLocker does not have access to any keys, and becausedata stored therein is encrypted, an EdgeLocker needs support of someother content server. A by-product of this approach is that cached datais only valid for a given TLS session.

What is claimed is as follows:
 1. A method of secure content deliverywithin an overlay network having a set of cache servers, comprising:during an attempt to establish a Transport Layer Security (TLS) sessionwith a requesting client, determining whether the requesting clientsupports authenticated encryption; responsive to a determination thatthe requesting client supports authenticated encryption, and followingestablishment of the TLS session, receiving a request for a contentobject at a first cache server from the requesting client; forwardingthe request for the content object from the first cache server to asecond cache server; and receiving a response from the second cacheserver, the response having been generated at the second cache serverfollowing termination there of the TLS session, the response includingan instruction to serve the content object from the first cache server,together with information for use in constructing one or more TLSrecords at the first cache server.
 2. The method as described in claim 1further including: constructing the one or more TLS records at the firstcache server using the information received from the second cache serverand without access at the first cache server to a session key; andsending the constructed TLS records from the first cache server to therequesting client.
 3. The method as described in claim 2 wherein sendingthe constructed TLS records from the first cache server to therequesting client includes sending a Message Authentication Code (MAC)received from the second cache server.
 4. The method as described inclaim 3 wherein a constructed TLS record includes the MAC outside ofencrypted cipher text.
 5. The method as described in claim 2 wherein thefirst cache server constructs the one or more TLS records using one ormore encrypted fragments associated with the content object and withoutsession-specific information about the session.
 6. The method asdescribed in claim 1 wherein the first cache server is positioned at anouter edge of the overlay network, and the second cache server ispositioned upstream of the first cache server at an inner edge of theoverlay network.
 7. The method as described in claim 1 wherein the firstcache server does not perform HTTP request processing.
 8. The method asdescribed in claim 1 further including maintaining a persistent secureconnection between the first cache server and the second cache server.9. The method as described in claim 1 wherein the information comprisesa sequence number, and a message authentication code (MAC).
 10. Themethod as described in claim 1 wherein the authenticated encryption isan encrypt-then-mac function.
 11. The method as described in claim 1further including: associating the first cache server of the set ofcache servers with a first security model; associating the second cacheserver of the set of cache servers with a second security model, thefirst security model having less tolerance of key exposure as comparedto the second security model.
 12. The method as described in claim 1further including: caching the content object at the first cache serveronly in an encrypted form.
 13. A computer program product comprising anon-transitory computer-readable medium holding computer program codeexecutable by a hardware processor to secure content delivery within anoverlay network having a set of cache servers, the computer program codeconfigured to: during an attempt to establish a Transport Layer Security(TLS) session with a requesting client, determine whether the requestingclient supports authenticated encryption; responsive to a determinationthat the requesting client supports authenticated encryption, andfollowing establishment of the TLS session, receive a request for acontent object at a first cache server from the requesting client;forward the request for the content object from the first cache serverto a second cache server; and receive a response from the second cacheserver, the response having been generated at the second cache serverfollowing termination there of the TLS session, the response includingan instruction to serve the content object from the first cache server,together with information for use in constructing one or more TLSrecords at the first cache server.
 14. The computer program product asdescribed in claim 13 wherein the computer program code is furtherconfigured to: construct the one or more TLS records at the first cacheserver using the information received from the second cache server andwithout access at the first cache server to a session key; and send theconstructed TLS records from the first cache server to the requestingclient.
 15. The computer program as described in claim 14 wherein thefirst cache server constructs the one or more TLS records using one ormore encrypted fragments associated with the content object and withoutsession-specific information about the session.
 16. The computer programas described in claim 13 wherein the authenticated encryption is anencrypt-then-mac function.
 17. An apparatus, comprising: one or morehardware processors; and computer memory holding computer program codeexecuted by the one or more hardware processors to secure contentdelivery within an overlay network having a set of cache servers, thecomputer program code configured to: during an attempt to establish aTransport Layer Security (TLS) session with a requesting client,determine whether the requesting client supports authenticatedencryption; responsive to a determination that the requesting clientsupports authenticated encryption, and following establishment of theTLS session, receive a request for a content object at a first cacheserver from the requesting client; forward the request for the contentobject from the first cache server to a second cache server; and receivea response from the second cache server, the response having beengenerated at the second cache server following termination there of theTLS session, the response including an instruction to serve the contentobject from the first cache server, together with information for use inconstructing one or more TLS records at the first cache server.
 18. Theapparatus described in claim 17 wherein the computer program code isfurther configured to: construct the one or more TLS records at thefirst cache server using the information received from the second cacheserver and without access at the first cache server to a session key;and send the constructed TLS records from the first cache server to therequesting client.
 19. The apparatus as described in claim 18 whereinthe first cache server constructs the one or more TLS records using oneor more encrypted fragments associated with the content object andwithout session-specific information about the session.
 20. Theapparatus as described in claim 17 wherein the authenticated encryptionis an encrypt-then-mac function.